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ABSIfiJLCT ^ 

This study vas conducted to ravietr avaluation studies 
of high school experiential learning programs tc detetaine the nature 
of their findings and the essential eleaents that produce effective 
learning experiences in the ccaaunity, and to uMBmBM the approaches 
used in these evaluations. literature reviewed for the study centerefl 
on <1) work experience and training prcgraas, c2) career or 
vocational programs, and f3) vocation al/noncareet experiential 
prograas such as service learning prograas. Findings froa the review 
of the literature indicate that the majority cf evaluations appeafl^to 
he poorly conceptualised and fail to build on previous evaluations of 
siailiar prograas. Based on these findings, a new frajiework for 
future evaluations was developed that includes three critical 
aspects: predesign investigations, designing the evalaaticn, aad 
iapleaenting and reporting the evaluation. CHiae policy 
recoaaendations that emerged in the process of aaseasing evaluations 
of experiential learning projects are included. I <liA) 
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PREFACE 

lliia research study prepared for thie Office of the Assistant Secretary 
for Poli<^, Evaluation and Research^ U. S. Department of Labor by staff 
of the Education and Work Program of the Northwest Regional Educa<:ional 
Laboratory reviews findings from a number of evaluation studies of high 
school e3(periential learning programs and proposes a new framework for 
future evaluations that appears more congruent with th^j nature of 
escperiential learning programs. As used in this study, experiential 
learning refers to a process of learning gained from both planned and 
unplanned experiences which involve the learner in me^uiingful activities 
and relationships with adults in the community. The learner is helped by 
another person to examine the meaning and implications of these^ 
experieiK^es for his or her future growth. 

Three categories of 'programs were studied: work experieiK» and training 
programs, career or vocational education programs and 
ncmvocaticmal/career experiential progrions sudh as service-learning 
programs. 

The results of the study are summarized in nine policy recommendations. 
These recommendations are practical suggestions that DOL's Office of 
Youth Programs and those involved in the planning, inplementation and 
evaluation of YEDPA projects can use^ 

• to direct researcdi into areas likely to add to knowle^H^e abamt 
haa to enhance the development, of disadvantaged youth. 

• to refine guidelines for project evaluations 

• to increase the quality and utility of. future evaluations. 



This study was conducted to review evaluation studies of high school ^ 
experiential learning programs to deter.«ine the nature of their findings, 
the essential elements that produced effective learning experiences in 
the coiMBunity and to assess the approaches used in these evaluations. 
After examinii^g the limitations of these evaluations, the authors 
developed a net? framework for future evaluations that appears more 
congruent with the iiature of experiential learning programs. I^e 
resulting framework has the potential for improving the quality of future 
research and evaluaticm of conniunity-based experiential programs where 
academic aiid/or career develo^ent outcomes are sought* 

The primary methodology of this study was a ccanprehensive literature 
review pertaining to evaluation studies of experiential learning 
programs. The literature review centered onx 1) work^-experdence and 
training prairams, 2) career or vocational programs and 3} 
vocational/noncareer experiential programs such as service learning 
programs* Since hundreds of es^riential programs have been evaluated, 
the authors frequently reviewed studies that synthesized evaluations 
across more than a single program. Tediniques fw obtaining evaluation 
studies from the literature included a computerized search of the ERIC 
datA base, contacts with federal agencies such as the Department of 
Labor, Hatlonal jCnstitute of Iducation and United States Office of 
Education, and contact with other research and development organizations 
stfch as Youthwork Inc. , National Center f<x Research on Vocational 



Education^ Center fr^t the Stu<S(y of Evaluation and Centejr for Youth 
Development and Research* 

This report contains three sections* Section 1 CKmtains our review of 
the evaluation literature of experiential education programs and a 
sunroary of findings. Based on limitations of prior evaluation studies 
propose a new framework for future evaluations in Section II. Policy 
implications from this study are found in Section III. 
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I, LITBRAtrORE REVIEW 

A nunbttC of writers hav* advocated the use of e3q?«>riential methods In 
education. Significant amcmg them have been the "raid-seventies* reports 
e.*!. Coleaan et al.. Brown et al., Martin et al,, Weinstock and American 
Youth in the Mid-Seventiea (cited in Zajchowski, 1978) , William Van Til 
(cited i^ Zajchowski) has provided us with a succinct summary of these 
reports t 

♦ 

The mid-1970 *s high school reform proposals encon^ss a variety of 
problems and are naturally different froo* eadi other to some'^ree. 
In general^ however, the reports are critical of conteniporary high 
schools as too large, age-segregated, overly separated, 
quasi-custodial enviroiunents for adolescents whi<di do not 
sufficiently provide youth a transition to maturity and the adult 
world <^ wMTk and ooramunily participation. In general, the reports 
support smaller, more diverse, age-integrated, and coonunity-related 
schools characterized lay both academic and action learning and 
supplemented fay alternative pat±s to maturiV through eaq^eriential 
learning 1^ way of business-industry work organisatioiui, social 
involvement opportu|^itie8 and coiraiuni^ education centers (p. 11) . 

Bven idiil^ advocating esiperiential forms of education, Zajchowski (1978) 
points to American Youth in the Mid-Seventies for criticism of the lack 
of research and evaluation (p. i.1). 

♦ There^ is nqt yet much en^irical evidence that action-learning 
produces significant affective growth in teenagers. 

. 8 ' 
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• is iittl« evidcokce that action^-learniiig ^oduces ^an 

iacrea8«d 8«n«ft of 'civic respoasiblXlty {p* IX). 

SiAilar rec(»endations have be«n made recently (Schwondt, 1978 1 Keeton^ 

ft 

1978) that, in order to maintain credible standing, educational 
institations cannot continue to provide credit for experiential learning 

» 

without demonstrating the effectiveness of es^riential edu^ion 
programs. Programs idiich are government sponsored have alrea<3ty begun to 
feel the pres,«ur« (Schiller, 1978). Congress is demanding evidence on 
which to base billions of dollars mtth of decisions— which programs 
should be kept and for which groups? 

Experiential learning programs may be clarified in a number of ways such 
as the type of setting or the purposec of the program. The experiential 
programs discuased here will be classified as work ea^cience and 
training, career/vocational education and nonvocational/career 
ei^riential programs, this grouping is baiMd prim»ily on th* various 
outcomes studied. Work ^sgperience programs are generally evaluated in 
terms of economic variables— %^ther participants are mc»:e likely to get 
a job and to make higher wages than if they had not participated. 
Career Aocational education programs are often evaluated primarily in 
terms of outcomes such as career knowledge and work attitudes in addiction 
to affec^ve variables— principally self-concept. Other typ9» of 
experiential programs have usually been studies concerned with assessing 
self-concept social responsibility and affective growth of participants. 



Work Bcp»ri»i>ce and Training Prograaa 

liork exp«rienctt and training prograns focus on vocational akill training 

and irarticipants are typical^ paid for their involvement. Many 

--^ ■ • - 

work experience and training programs are aimed at the eccwiomically 
disadvantaged and have had the overall purpose of decreasing unemployment. 

Schiller (1978) has reviewed studies relating to federal mai^wer 
programs. He found three major studies of the Public Bi^loyment Program 
(PEP) which became part of the Comprehensive Education and Training Act 
(C£^) programs* 

PEP prograift data do not indicate positive earnings effects. 
Conquer isons of pre- and post-program.participant data show a slight 
decrease in hourly wage rates. Bnployment increases revealed in 
program data are easily eiqplained. The program enrolled persons of 
near "average" characteristics when they were underwnployed or 
unemployed! later these persons regressed toward the mean <which was 
likely to be employment) (p. 112). 

I4iree other programs which Schiller also di&cussed included Operation 
Hainstream, WIN and Supported Work. All had single evaluation studies. 
Hone of these studies showed higher employment for enrollees following " 
participation. Bowever, tha evaluation of the Supported work program did 
not yet have postprogram res%ats. 

Barton and Fraser (1978) also reviewed a stu*^ of the Will project 
completed in 1975. fhis stud|y found no succ&ss-rate differences among 
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enrolleft charact«cistics rac* and sax) txctpt that a highar lawl 

of aducatloQ .appaarad to ba asaociatad with both high placamant and low 
dropout rataa. Siort-tacm job placamant affacts of tha programs did not 
hold in tha long run. ^ 

Sailllac discussad tha findings in savaral stadias that ttere ara 
diffarantial rasolts across programs such variabXas as racst 8«x and 
prior aduca^ion laval* in a study hy SchiUar and othars (Schillar at 
al^, 197e) ha concludad that thesa apparant diffarancas'could ba 
explainad "on the basis of recent preprogram enploynent experience (no 
ei^ploymant varsua soma amploynant in the prayious six month period) It 
woul^'-^J^r that evaluation studias need to clarify tha ralationship 
batwean client profiles and typa of program, axpariancas. 

' ' I 

Scdiiller also addressed tha Issue of so little dafinitiva evidence 

can ba provided on manpowar programs* lha first tiio points in his 

critique hold true for axpariantial programs in genaralt 

Thara hava baan hundreds of evaluations of specific manpower 

pcogrMSy and thasa provide a foundation for studias of ralativ« 

effeetivaness. Unfortunately, this foundation is itself not vary 

solid, as other rasearchars have noted. ' Tha most serious and 

familiar shortcomings of past evaluation studias ara: <1) tha 

absence of a coo^rison (control) ^rovp for identifying aat program 

affectsi (2) restricted time horizons that preclude naasuremant of 

longitudinal impact. Only a handful of past studias hava bean 



reasonably successful In ovarcoMing these problems and thus provided 
neanlagfal estimates of program ii^pact <p. 1U)» 

Barton and Fraser <1978) reviewed sttidieii of pzograms in "Employment and 
Training" which corresponds to the category of work Experience. The 
Neighborhood Ycuth Corps (HYC) model proviae* youth iaschool, summer and 
out-of-s<dKx>l options and in rural and urban settings. The "explicit and 
implicit* goals of N3fC "can be suramaf iaed as foUowss (a) to. 
redistribute income to the poor, (b) to increase the eiDployment of youth, 
(c) to reduce teenage related crime and Cd) fe> increase the lifetime 
earnings of enrollees through training, incentives to stay in school and 
* work experience (p. 68)." . » 

Of the six studies cited, only two had oonpariscm groups for determining 
program effects. One of these 8tudl<::s found that participants and an 
older iK>rking comparison group maintained their functioning in terms of 
grades, good self-conc^t and "sense of control of their own destinies." . 
A younger control group and dropouts twanged negatively on these 
/mriables. Little evidence was cited to d^nonstrate enhanced postpcogram 
cnployability or earning gains of participants which was the principal 
program objective. It was not clear whether NYC helped students to 
'^aduate from high sdiool, . There was evidenct from one studty that 
effects may differ by such factors as age# race, sex and school status. 

Barton and leaser report another study which evaluated the two-year 
Vocational Biqploration in the Private Sector program. This stu^ found 
that. While both grade point and attendance increased significantly for 

/ iz • 



participants ov«t cooqparison gto^p students, th« proportions which 
^rsduated or dro^pod out rsmalned the samft* following gcsduationr ths 
coiAparison groc^ showed twice the rate of unoiiploynent that the 
parti.ei{>ant groi^p did. An evaluation of the Youth Conservation Cor{ui 
showed that the young people involved were generally quite positive about 
their eacperiences. White and Spanish American oorpsm^nbers were moi.e 
enthusiastic than Black and Native \amer leans and older (18 year old) 
corpsraembers were more positive «ian younger (15 year old) corpsmembers* 
Goldberg and associates (1978) studied the noneconomie effects of the Job 
Corps, namely knowledge of work, attitudes toward %#ork, in^i^rest in, work, 
employment status and social-attitud:^i^,.ln^acts. Groves which were 
ccRQpared Included No Shows, persons accepted into Job Corps but- vrho 
did not participate (used as a comparison group) i Persistera, those who 
completed the course? and Dropouts, those who terminated prematurely from 
the program. 

Findings |n the airea of knowledge of work were that those who remained in 
the laborN^^ket (No Shows) did better thai^i Persisters, who in turn 

I 

Showed more Improvement than Dropouts* Attitudes toward work did not 

♦ 

change for No Shows, were mixed with one significant decline for 
Fersisters and declined significantly on all three measures for 
Dropouts. Bi^loyment status also failed to show advantages for 
Corpsme^ Although Persisters tended more often to be eiq;>loyed fulltl^ 
than others, the percentag4» of time worked after Job Corps termination 
was signifloantly higher for No >Shows. Women tended to benefit more. than 
men. The most apparent finding was that Dropouts shoiMd^^be^ia^ 



A69atively on all indicator*. In the area of social-attitudinal changes, 
Persisters ifl«>roved on all five indicatncs while neither of the other two 
groins made substantial gains, Pemaie Per sisters, especially, showed 
gains in selfesteem* 

Another area of inquiry for this , study was tiie health and educational 
itapact of Job Corps, Pcrs.isters did not improve on nutritional 
information cut were significantly higher on the nutritional behavior 
measure* Health care (except for glasses) was better for Persisters 
since the Job Corps provided free care. In terms of educational 
attainment, it was found that few Job Corps m^bers (a) received iSEOs or 
{b) were ever enrolled in Job Corps classes. •Female Persisters, 
however, were the most likely to be in those two categories. Women of ' 
both groijps «^ntered Job Corps with more educational backgrouna <p. 16) 
Some, insist into the problems of the Job Corps educational coi^ponent may 
be gotten from a recent evaluation of the World of wdrk curriculum 
component (Mellon, 1978) . This evaluation was based or an examination of 

* * 

the curriculum objectives, materials and outcome measures. She 
•valuators found a lack of performance actives and good outcome 
measures. Tliey recommended that current World of Work materials be 
revised and some available conmercial materials be incorporated. In 
addition, a recommendation was made to include an experiential^componeijt 
to supplement and reinforce the listening/reading acquisition of 
knowledge. 

J 
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Summary of the Work Experience Programs 

Despite the tr^nend^s investment in tha Work Experience and Training 
programs, littXe evidence has been found that the long-term employment of 
participants is positively affected. Other indices, such as attitadinal 
and self-concept measures have, in a few studies, shown some positive 
results for youth who remain in the programs for the complete cycle. 
Such aianges cou?.d affect the employability of the participants. In 
addition, the data whi<di do exist suggest that prograamnatic effects may 
differentially affect various subgroups in terms of race, sex and 
previous education. Finally, two studies noted that the greatest gains 
in both eii5>loyability and attitude ma^ be within "hard-core* groups such 
as offenders and Black teenage f^ales. ^This type of variable must be 
studied carefully since these groins may have nowhere to go but "up* and 
the findings may be partially due to statistical artifact. 

Career/Vocational Education Pro<irams 

Career education programs in this report are, for the most part, school 
based. They generally focus on helping participants acquire career 
knowledge and decision-making skills— skills which will facilitate job 
seeking, job maintenance and career planning. Some of these programs 
include vocational skill training but it is iiot the principal program 
purpose. Students may or may not be paid for their activities but 
receive academic or vocational credit toward graduation (at both the 
secondai^y and postsecondary levels). These programs usually attempt to . 
relate and/or integrate the career ei^riences with academic «q>eriences. 
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Bonnet (197^) reviewed and synthesized findings from 45 career education 
evaluation studies conducted in 1975-76. These included 26 projects at 
the end of three-year funding under Part D of ?L 90-576 and 19 projects 
which had received one-year grants through the U. S. Office of Education, 
Office of Career Education (OCE) . Although the instructional strategies 
were adxed, most of the projects claimed to be based on some type of 
classroom infusion model* To the extent it is possible to determine, 
only secondary level findings will be mentioned here. 

Bonnet discussed the evaluation outcomes within the ccmtext of the goals 
of career education set by the Office of Career Education: 1) Bonnet 
concluded that there is no evidence that career educaticm either 
positively or negatively affects basic skills (reading or math) 
adhiev«&ent. 2) Attendance increased in some cases but was not 
maintained during the second year. 3) Bonnet concluded that "...it is 
clear that the goal of fostering a desire to work is attainable through 
existing career educaticm practices." 4) Twenty-two percent of the 
secondary studies showed significant differences (p<.05) between pre and 
posttest means on measures of decision-making, jois-hunting and 
job-getting skills. 5) Job specific occupational skills were not 
generally assessed. Interpersonal skills were usually evaluated using 
self-concept instruments. 6) Self-understanding was measured by 
self-esteem ratings and items which related to self -appraisal in terms of 
career decisionmaking su^i as are found in the Career Maturity 
Inventory, lliere were few significant differences found at the secondary 
level for either the self -esteem or self -appraisal studies. However, 

. 16 
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evaluation of career knowledge showed career education to be 
"conclusively successful" in this area. 7) Mo evidence %«s available to 
determine awareness of continuing or recurrent educational 
of^rtunities. 8) There was no evidence on whether gradua^tes were 
actively seeking or placed in career or educational situaticms consistent 
with their career goals. Most graduates had had only limited exposure to 
career education at the time of the reviewed studies. 

Another recent synthesis of career education evaluations for the Office 
of Career Education (Bhaerman, 1977) addressed the issue of basic skills 
achievement. Bhaerman's report stresses that, while all studies did not 
see gains in basic skills achievement, many did and few saw negative 
results. 

Experience-Based Career Education (EBCE) was developed four regional 
laboratories beginning in 1971 (Bucknam, 1976 1 Bernhardt and Owens, 
19^9). Each of the resulting four models had certain cannon progran 
characteristics: community sites served as the focus for student 
learning experiences; each student had an individualized program based on 
their specific academic and career needs; and academic learning was 
integrated with career learning. 
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A synthesis of secondary jLevel pilot site evaluations through 1976-77 
(Bernhardt and Owens, 1979) showed similar findings to other career 
education programs. 

• PBCE students progress in reading and mathematics at a rate at 
least equivalent to the progress of non-EBCE students. 1 

• Significant growth in career attitudes and career knowledge was 
detected at most sites by a variety of career development 
measures. 

• Significant increases in student attitude toward self and others 
were found at only a few EBCE sites (p. 15-16) . ^ 

In addition these evaluations assessed the attitudes of the students, 
parents, strff and community site supervisors toward the es^erlences 
students have in the programs. The c^inions of these groups were 
overwhelmingly and c<mslstently positive that the es^eriences are 
worthidiile. 

t 

The Career tntern Program <CIP) was funded by NIE and iropl«nented in 1972 
with the Philadelphia schools, fills project enrolled students In an 
alternative school who were "not succeeding In their 'home* high 
school." The program featured individualized academic planning. 



^These findings were also Included in the Bhaerman (1977) synthesis • 
reported abo^'e. 
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coursework, personal and career gcidance, career eaiploration and 
^ostprogram j^ac^ent. This program vas extensively evaluated. 

• CIP interns showed higher school completion and lower dropout 
rates than contix>ls. 

• CIP interns showed increased responsibility on factors sucdi as 
at tendance t keeping aj^intnents, rescheduling appointments and 
completing assignments. On a negative note, the stu^^ conceded 
that those result were wa at some' cost in staff time and 
effort. 

• Regarding test taking skillss "in omparison to control 
students who *»ere not enrolled in CIP, interns learned to listen 
to instructions, to ask for relevant information and to use the 
time available in a task-oriented way (p. 130).* 

«> 

• Interns broadened their use, of resources, including peer 
resources « 

• Class participation increased both in quality and quantity* 

• "Successful CIP interns are willing to repeat courses they have 
failed or to review material they have already studied 

(p. 

• Reading and math achievement increased significantly for interns 
over <x>ntrols, although they r«naiTOd below national averages. 

• Interns did not increase significantly over ccmtrols in terms of 
a sense of self-iiorth. 

• "In both attitudes an^ knowledge thought essential for career 
planning, interns* scores increased substantially more than did 
controls (p. 147)." 

i9 
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Barton and Praset (I97e) revimd programs they termed "occupational . 
education and learning through esq^erience.* Ilhelr review of vocational 
education is included in thiis secticai. The trend of these programs has 
been toward an integration of vocational and acadenic educati<» which has 
been much influenced by the career education movement. The Vocational 
Education ^t of 1963 and the Vocational Education Amendments of 1968 \^ 
provided the tn^tus for numerous vocational education programs. Barton 
and Praser cite several sumnaries of both secondary Mid postsecondary 
vocational education program evaluations spanning the time from 1963 to 
1975 • The ccmaensas of these studies is that dropout rates are higher 
among vocational education studentsi there is little or no en^loyment 
advantage for vocational education program graduates; those who- trained 
in highly specialized or technical fields seldom got jobs in those 
fields; a student *8 socioeconomic bacicground rather than the type of 
education received (other than college graduates) may be. the single most 
important factor in, their employment and earnings potential. 

Cooperative education is another program which has also benefitted from 
federal legislation at both the secondary and posjbsecondary levels. A 
study by Frankel C1973) provides* an interesting oonparison-hetween ^ 
cooperative education, career escploration and work-study programs. ^ 

2to the extent that these findings compare cooperative education and 
work-study programs, they nay be viewed as reasonably valid, however, 
the careet education programs w^re considerably fewer in number, (and 
probably less representative) than programs in the other two categories. 

20 
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^•eif ically» it was found that a coo{»ecative dducation program ia 

• « 

»ora likaly than any othar typa of progruA tot 

• Provide studants witlv job-r41atad instruction in school 

• Hava a foUowup pxogram for its graduates 
a Hava an advisory committaa 

a Provide job placement services 
, a Have a high rate of job-related placements 

a Provide students with jobs that offer formal on-the-job 
training 

a Help students In deciding on an occi^ation 

a Provide students vith j<rft>s that fit into their career plans 

• Provide students with ^obs^that have a high level- of 
responsibility 

a Provide students with jobs «»at afford a high degree of 
satisfaction . 

♦ * 

FeoB a negative stand^int cooperative programs, when ooiqpared to the 
other types of work education programs, are most apt to discriminate 
against studants on the basis of student attitudai they are lass 
afCactiva in reducing student absantaaismi and, because they place 
students in mora responsible jobs, they are mora apt to interfere 
with a student's othar activities sucii as school work, dating, 
sports, etc, (pp. 3-4). 

A mora recant study ty Cohan and Frankal (1977) of postsacmdary 
•cooperative education programs found that co-op graduates had higher 
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earnings than nonco-c^ 9»duates. This find'ing did not hold if only 
females were <:H3nsideredi iJob eai^erience a|ipeared to be a heavily 
contributing factor to the greater salary of co-op graduates. Students 
wre found to have benefitted financially airing school and the co-op 
experience was viewed as "an important financial aid device for the 
particip .nt* * 

Another important conclusi<» was that 

Pive-year programs provide significantly higher annual student net 
incomes ttan four-y<-ar programs, and both provide significantly 
higher student net incomes per year than two-year programs, tftiis 
• pattern holds for every net income subcon^nent, except for- tuition 
where four-year programs have the highest per-year values (p. 11). 

In terms of pon-economic outcomes th^ authors noted that 
Graduate ei^riences relative to job satisfaction, job 
responsibility, work continuity and career plans were also examined 
in this mtvSay* Mo job satisfaction differences were statistically 
significant, nor were differences in job satisfaction observed wong 
other sasple stratifications (e.g., sex, race, GBk, program length, 
discipline area). However, some differences in levels of job 
responsibility were identified by co-op status. Xh^se differences 
indicated that co-c^ participation leads to somewhat greater job 
responsibility upon graduation, this is consistent' wi^ another 
fining that oo-<^ participation is treated, in many re^>ece8, as 

similar ^to previous ei^rience by the employer in terms of earnings 

\ 

(p. 11). , ' 

22 
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Thft najor ar«a of consensus among other studies cited Barton and 
Fraser was that cooperative education graduates had a distinct advantage 
in offering eni>lo:ynent based on their work experiences, often with one of 
the cooperative employers. The obverse of this finding was also true, 
that en^loyers found cooperative education programs to be relatively 
ines^nsive recruiting grounds « 

The Executive High School Internship program places students into 
management level internships four days per week for a sweater. One day 
is spent in a three-hcrvr seminar and performing program housekeeping 
tasks. The Crowe and Walker evaluation (1S77) used a comparison group 
which had volunteered to participate at a later time ««n the study 

participants. T^aeir groups proved, however, not to be strictly 
comparable. Variables studied included knowledge of organisational 
theory and behavior, the management function, work habits, personal 
growth and writing. None of these variables Stowed significant growtJtl 
pre to post nor participants versus confer ison. The knowledge and 
writing areas proved not to be directly addressed fcy program activities. 
The failure to observe change in personal growth areas and work habits 
was attributed to t^ fact that these traits were in^licitly used as 
participant selection criteria. 

i 

* « 
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Barton M Ftaser discuss several otber career education types of 
programs noting that evaluations axe «ctreme3y pbor or nonexistent. 
«»«e prograns iijclude the e^cperiaental Work Bx||srienoe and Career 
Exploration trogran (WBCEP) , Alternative Bducaiilsn ar. J Oomounit^ 
fiducation. j 

I 

Suamatv of CareOT/ffocatlontl Mueattoa Prairan* | 

Career education prograns generally result in gif^twr career knowledae 

» .1 

and positive attitudes on the part of participants. These programs, with 
the exception of the Cooperative Education programs* generally do not 
resist in greater postgraduate emplo^ent for participants, 5he 
long-term effects of these types of programs have not yet been thoroughly 
assessed*^ One aspect n^ich was not discussed in af^ of the 
evaluaticHis was the relative effect ot the programs on various population 
subgroi^s (e.g. l^T race, s«c, etc.). t 

The relationship of basic skills achievement to participation' in career " 

If 

edu<»tiQn programs mas studied in several of the sources, vsited. - ahe 
evidence includes several situations in which career education students 
gained in reading and math over control students and-few in wfaicdi th«y 
daclined. A large number of studies found no differences and the 
consensus appears to be that no adverse effects should be ei^ected. 



^Biester (in process) describes a recent effort to conduct a three-year 
f olloirqp study of ffiCB students* 
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MonvoMttional/Car— r Bxp»ri»nti«l Proorama 

tin this c«t«9ory o£ sa^ftri^ntiaX «dacatlofi jprograms at* Inc^uaetethom 
ubieh ar* mt focaf«d toward vocational skills or <^^fim^l^m 
acqaisitiofi. Hatlonaiay aiss^nfU^d programs in this catagory inclu^ 
Sarvice-tearaing, Outward Bound and Foxfire (cultural journAlisro) , 
evaluation data are particularly scarce for these programs partly because 
the objl^tivS^tend to be affective and partly because their development 
iiias generally not federally funded. 

HOose (1974) studied 56 students who had narticipated in postsecond?iry 
service-learMng Interships in North Carolina from 1967 to 1971. Hoose > 
describes the internships as follows s t^ical service^learning 
internship is a twelve-week, full-^time assignment with a public agency 
through which the student receives acadendc credit and stipend (p. 
Boose interviewed 20 and surveyed 36 former service-learning interns* 
She found tiiat i^roxinately half were working in public service jobs, 
.Where academic credit' was awarded, there was apparently a greater 
degree of learning outcomes realised by the student.* A conclusion based 
on the interview alone was that "Saving independence and public 
responsibilities while serving as an intern were in^rtant factors to 
many of those students.** 



\ 
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«ducatifiii^ in 20 inaejpendttnt secomaary schools* Questlbanaira mxm 
sdninistftrsd* to partiotpuiU (pir<»*^8t), oon^arispn students, faei^^ 
parents, cottmonity supervisors, school office petsonael and i^ogran 
directors* Because the stud^ has several methodological probl^s, the 
data nust be considered with caution. Rowever, some of the author's 
conclusions are 'interesting t ' 

The determinants of participation were not also the determinants 

successful participation and while academic factors may 
infiWnce participation, "it is social, and personal 
charact^H^tics *Aicii are associated wi^h sat'isfaction in the 
endeavor** 

Students without\ef inite c^eer or' educational plans tended to 
be less successful thaiiYllfose-^ 4iadjnore definite goals. 

".v.. • k' 

Personal development was Sej^i as both an ^mport^nt goal and 
outcome of experiential progn 

Success" raUp corresponded to increaSM hours In the program 
with regular and continual participation mm S^ortant than 
intensi^. . • \ «• 

Hiose who moved to a i^ecial living situation were far more 
likely to have a successful experience than thove who remained 
at home or in a term on campus. 



\ 



'""Off-campus education" apparently included d^reer education types of 
programs. Rowever, the t^^ of programs and t^ Relative participation 



were' not reported. 



<- • Cottns«iio8 oc •debri*f i»9" i« «8Soctattt4 yri%lk nor* highly 

8uce«8«a]l «xp«ri«iM:e«« tn addition, one-to-on» convatsat^oas'W 
idth faoi^ IMS ^parafit2y tkof he^jpful th«» stminar-l^ ' 
sessions oth«r participants. Tha author ^cuXatas that 
"tha student \matm to prasarve tha uniqueness of the.ea^arienca 
and find its special meaning for hiA or her." (p. 25).* 

Shore iimi ooiqpXeted an extensive sunnary of research and evaXua^on 
studies of Outward Bound Programs. Two principle types of Variables were 
studied with regard to Outward Bound Programs— self-concept and off#iider 
recidivjlsm. The auth<»: includes self-concept, self-estesm and 

* * 

self-efficacy all within the c&tegory of ^ self -concept; Baving done so, 

he concludes that the studies *do not suggest consistent, stable, \ • 

replicated results with regard to tn^ effect <^ Outward Bound on : 
self -concept (p. -53).* Bis conclusions concerning recidivism are more 
positives -*Studty results on^ the effects of Outward Bound on recidivism 
seem generally conslsient.** and "yT^sitive evidence seams to be 
mounting, though it is fay no means ocmcluslve.** (p. 53|." Shore is 
extremely critical, of methodological flaws and lack of followup in most , 
of the studies he reviews and, tharefm, cautions against 
overgeneralisation. . 

Conrad and Min reported the preliminary findings for ^ 

BvaluaUon of Bi^rientlal Z«arning' Project (EBLB). in the initial work, 
24 possible outcomes wp^ listed and 4000 students in 20 programs- were- '^^ 



a.:;^ to rapoad on a four-point scale the extent to whl<^ they felt 
their program helped th^m to develop in. various areas* Fourteen of the 
item received 80% or higher agreement. These il^ms weret 
1. cincern foe fellow human heings^ 

2* Ahillty to get things done and to work smoothly with others 

3. Realistic attitudes toward <^her pec^le such as the elderly^ 
handicapped or government officials 

4. Self*motivation to learn, participate, achieve 

5. Self->conoept (sense of contisdence, sense of competence, 
self-awarene^} 

. €• Responsibility to the groiqp ^ clas^ 

7» Risk-taking— c^nness to new l^cperiences 

1 

8. Sense of usefulness in reljati^ to the conmunity 

9. Problem-solving ^ 

. 10, Risk-taking—being assertive a^ independent 

^ 11. Accepting oo^isequenoes of iqy oi^ actions 

12. Gathering and analysing information, observation, reflecting on 

• ej^erience . \ 

\ • 

13. Knowledge of •omimuBiity organizat:^ons 



14. :tosponsibility for my own life (pi. 28) 



l!he authors conclude that "the ear^y findings of EELP suggest that direct 
ooMunity ei^rlenoe may be..an iiqportant med^is for nurturing certain 
kinds of growth and develcsnent in students. !lhat similar findings came 
from such a diverse range of pro-ams— urban,\ rural, mnall town, private,' 
p&rochialv public— is especially significant. i (p. 29)^" - ^ • 
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B«ckin (in process) cites several other studies of the "service-learning" 
tvpe. Two studies found that college students gained significantly in 
self-esteem after performing volunteer vork with retarded children. 

Another stu^ reported that students who were involved in political or 
social action outside the school became more open-rininded and politically 
efficacious. Black and White high school students who did field work on 
the political structure of their city increased in feelings of political 
effica<gr; however. Black and Chicane students in another school did not. 
Other equivocal findings included no significant differences on political 
and social efficacy measures between a group %^ich performed volunteer 
service and a caa^aris<»i group, ^ ^ 

Another program whi<^ falls into the nonvocational/career category is 
Fe»cfire or cultural journalism programs. ^ In these programs students 
do the research, writing,, photography, layout required to publish a 
magazine generally devoted to interesting persons and communis history 
(Sitton, 1978). The programs are generally viewed as interdisciplinary 
and contain elements of (at least) Stilish, journalism and social 
studies. Although there may have been Itjcal evaliiaticns of cultural 
journalism programs, there are no published evaluati<»s available at this 
. time whicii provide an assessment of the impact of these programs on 
students. 



^Foxfire is the name of the program at .JUd>un Gap, Csorgia and, 
although the program concepts have been disseminated, adoption sites are 
generally termed "cultural journalism? programs. 
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SmwMury of >tonvocatlonal/Care»r Experiential grograms 

As noted in the introduction to this section, the variables **ich for the 
most part have been evaluated in nonvpcational/career experiential 
programs are affective variables presumably corresponding to affective, 
objectives of the programs. The weight of results must generally be 
considered inconclusive. Definitive results await additional and 
methodologically sounder studies to be conducted and reported. It should 
be noted that at least one of the major programs in this category, • 
cultural, journalism, has had no outcome evaluation to date. 

jponclusions 

Three categories of programs (work eacperience, career/vocational 
education and ncHivooaticmal/career experiential programs) were reviewed 
according to available evaluation data. a*ese categories -were based on 
program goals, the characteristics of the programs and the t^>e8 of 
variables addressed in the evaluations, in terms of general conclusions^ 
few definitive statements can be made based on the evaluations of these 
programs primarily due to general problems with tte evaluation studies 
which were indicated fay the various authc^ cited (evaluation study 
weaknesses will be discussed in the following section of this report). 

The following are statements simarizing the conclusions which can be 
drawn from the findings reputed. 

• There Is little solid evidence to demonstrate ijqprcved 

«««>l<jyability of participants in work ea^rience and training 

programs. 
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Work experi^ce and training programs may <Siif£«rentiany affect 
types of participants in terms of their race, sex and previous 
education. 

Soroe evidence Indicates the most positive effects of wrk 
esiperience and training programs msy be with the hard-core 
si^roi^s such as offenders or Black teenage females whose entry 
scores are initially lowest. Such effects may be real or m«^y be 
statistical "regression to the mean." 

Career education participation results in greater career 
knowledge and more positive attitudes toward work, although no 
demonstrated increases in employability. The exception to this 
latter statement is the cooperative education programs- that . 
oft«i result in students being hired by the en^loyer to which 
they were assigned. 

Career^ education does not adversely affect basic skills 
achievement and in some cases a^ears to be responsible for 
increasing tested achievement. 

There is some evidence that both work experience and career 
education type programs positively affect attitudlnal 
variables. However, to a large extent this haik not been studied 
in the work experience and training types of programs. 
Although this issue was mentioned in only a few instances, it 

« 

did not seem to matter whether or how much students were paid as 
to the attitudinal outcomes they evidenced. 
1!he evidence is inconclusive as to whether various 
nonvocational/career experiential programs positively affect 
such variables as self -concept or self --esteem. 
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11. P!«OPOSB>.mMBfQRK FOR FUTORB SVALOATlONS 

General* findings from a review of the literature indicate that the ' 
najority of evaluations ai^ear to be poorly conceptualized and fail to 
build on previous evaluations of similar programs. These limitations of 
prior evaluations have led to a proposal for a new framewrk for 
conducting such evaluations. Critical a^ects in this framework center 
around: 1) predesign investigations, 2) designing the evaluation and 3) 
implementing and reporting the evaluation. 

Ihis section of the paper presents a framework for evaluating 
«(periential learning pzograms that has evolved as a result of — 
investigating unique characteristics of experiential learning programs 
and numerous evaluations conducted in this area. IChe approaches proposed 
here are not meant to be another evaluation model but rather a 
recommended framework for consideration by people already familiar with 

» 

general evaluation principles. Key elements of this framework are 
depicted in Chart 1 and are discussed In the remaining section of this 
paper. 

Predesign Investigations 

Prior to developing an evaluation plan for an experiential learning 
program, it is useful to study factors both external and internal to the 
project. External factors include a review of some th«k>ries that may 
help organize and establish relationships among the separate objectives 
and student outcomes likely to be ea^loired in the evaluation. Sxanc^les 

4 
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Ch«t 1. rr««,i«* tor Bv.l«.tta9 Ew«i«tiU l.«»ia3 Srogrw. 



gredesiyi Inveatig&tion 



Evaluation Desig n 



1) Related learning, career 
development and human develop- 

'inent theories. 

2) Findings from similar prior 
evaluations. 

3) Instruments found useful in 
prior studies. 



^iIn«S ^^T^ '^f" assessing iearking- from both ' 
planned and unplanned stimuli, 

2) Identify relationships between short- ^d long- 
term objectives. ^ 

3) Plan to assess the relationship between 
ea^erxential learniiig processes, program 
delivery strategies and student outcomes. 

4) Consider how different learnings occur in the 
settSigs ""^^ community- 



Evaluation Implemen tation and Reportincy 



1) Determine the variables to be assessed. 

2) Dse evaluation designs appropriate for the 
developmental phase of the project. 

3) Use naturalistic evaluation techniques where 
appropriate. 

4) Report outcomes by characteristics of participants 

5) Include descriptive case studies to supplement 
statistical reports* 



of «iM3h thMTlM laeluat vocUX iMrniiig theory <Baiidura; 1979), 
AttribuaoR theory (Bar-Tal, 1978) and noUvatiem theory (Atkinson, 
1»€4) . A second area worth parsaing Is that of evaluation findings 
resulting froa i«ioc prograns similar to the one to be evaluated. In. 
this respect^ it is equally iiaportant to determine whi<A outcomes have 
been found not affected lay such programs as to find which have shoim 
significant changes. The third t^ of information useful befo re 



preparing an evaluati<m plan is the names and characteristics of 
evaluation instruments that have been used to assess outcomes similar to 
those ejjpected in the program to be evaluated. As with program 
objectives, it is important to know i^i^ evaluation instruments revealed 
significant student gains and which ones^did not. In the recent 
evaluation lynthesis of career education programs ^onsored by OSOE 
Office of Career Bducation during 1975-76^ Deb<»ah Bonnet (1977) 
identified, for example, a widely us^ commercially available instrument 
of career maturity that has revealed no significant growth in 32 out of 
34 evaluation studies (p. €0). 

Pesiqninq the gvmluatien 

It has become fashionable for evaluation critics to attack goal 
statements and behavioral outcomes of n^wy educational progrwns for lack 
of precision and coherence* Freeberg (1976), in writing about youth work 
training programs, has stated that "Instead of verifiable procedures, the 
prescriptions provided for specifying training objectives and associated 
measures appear to begin and end with purely consensual CMt rational ^ 
decisions, usuaUy leading to a laundry list of recoimended performance 
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ootoones tli^t aon«licM 9^til to ^« particaUr writer* (p. 537) m 
contrast* Preeb«r9 borrows £ro» criterion theory to rttcommend: 1> 
multiple- behavioral iadexes especially in the initial phases of criterion 
development* 2) ^specification of criteria along a ten^al continuum to 
include short-^tern, intermediate and long-term measures and 3) concurrent 
and predictive validity (the relevance of a criterion to others obtained 
at the sanie time and in the future. 

In evaluating experiential learning programs %Aere mu<A of the learning, 
takes place in an individualized manner at a community setting it appears 
important to not only assess predesigned criteria, developed in the 
manner proposed ty Freeberg, but also to.be alert to assessing learning 
resulting from ui^lanned events which are experienced fcy the learner and 
reflected upon hy the student and learning coordinator or teacher, An 
exan{»le of an ur^lanned stimulus for learning can occur in a situation 
where a student shows late' several da^s at a job site he or she was 
exploring and gets told Iqr the employer not to return. An open 
discussion on this experience by a student and learning coordinator can 
produce new insights to help the young person modify both attitudes and 
behavior* 

traditionally, ei^loynent ahd training programs have assessed «diether a 
student has gainad job entry skills, gained enployment in the particular 
occi^ation and the salary received. More recently, researchers have 
stresiBed the need for also assessing occupational adaptability and 
transferable skills (Pratsner, 1978) that facilitate a person's ability 
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to function succeAsfiiUy across « vart^^* of camrs* «ho«e Involvad in. 
camr «duoation are faced with broader outeonas tiiat include fa^roved 
iMlf-conc^t, %K»k values^ decision-making skills, use of leisure tine 
and awareness of means available for continuing and recurrent education 
(Boyt# 1978). Service learning programs have focused on helping students 
to learn concern ^oc others, ability to work with others,' realistic 
attitudes toward different types of people and self-motivation to lea^n^ 
participate and achieve (Hedin, 1919), Clearly the list is unending and 
thus a critical element of the evalua tor's role is to determine whitA^ 
objectives are central to the program under investigation and i^ich have 
at least a fair dbanoe of being achieved* 

Nhile it is important to look at learner outcomes, it is equally 
important to describe the delivei^ strategies being used to help students 
achieve these objectives. In the past, the delivery strategies have been 
generally ignored or described in broad terms such as career esqploraticms 
and career counseling. In addition to depicting these strategies in a 
more concrete way# it would be useful to assess the experiential learning 
processes involved. In a recent researai studty based on a social 
learning theory framework, Owens and Owen (1979) identified a nuntoer of 
aiaracteristics of ei^rl^ntial learning that high school students in 
eight B3qE»erience-Based Career^ Education projects identified af ia^rtant 
reasons ^ a particular ooomiunil^y experience was an excellent c^r poor 

learning ei^rience. Characteristics associated mpst -lirith excellent 

• ! 

learning ei^iences were, in descending order of in^rtances' 
1* Trying out taidcs- 
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2« Bfting giv«ii ml ni^poiisikility 

3* LUtening and tallciiig with peopl* connimil^ 8it«s 

4. R«cftivla9 el«air dtr«ctioRB to foUoir 

5. Engaging in challenging tasks • 
5, AiJplying learning to new things 

«* Adult ftucouragemettt for doing tasks well 

7. Learning tedjnical wc^ds and laniguage associated with a carear 
of interest 

8. Freedoa to e3cplor« ar«as not planned in advance 

9. Persofutl interest of a supervisor 

10. Sncouragenent fro« family or friends 

11. Knowing ahead of tine what would be es^ected 

12. Discussions of the 8tudent*s escperlences with staff and other 
studentui 

Characteristics students associated most with particular oonmunity ^ 
ea^rienoes resulting in little or no learning were, in deoending order 
of in^rtancex 

1. Boring tasks 

2. NO personal 4^iportunity to try out the mosk . 

3. too inidh ri^tition of ten activity 

4. Ho opj^tuni^ to esq^lore otheir areas oi interest . 

5. Mo opportunity to apply learning to new things 
€^ Dot knowing i^t would be expected 

7. Mo recognitiopk frcM other workers for doiag tasks well 

e. Mo ofiportuni^ to observe or talk to workers at the site 

• •• . 

• .... 
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9i Lack of clftar dirtotlont to follow 
10» MO <v^tuiiiti*s tq discu^ ay «]^rl«iicfts with othftr studtnts 
11% Being too closely suparvised . 

^cttived iinpottance of factors related to positive or negative learning 
ea^eriences was not affected lay student's s^. Bowever, a significantly 

greater proportion of youn? feiiuil#« i-hiin 4.<«,.t-tfi^^ ^ bettur 

understanding of others as a learning tKitoome fron their 1BC8 community, 
site tti^riences. Students of both sexes felt that a major outcome of 
their. ccMimunity experiences was a better understanding of a specific 
career of interest. 

More detailed research on some of these <Aaracteristic8 is likely to 
reveal important factors that iBotivate a young person to learn and ta> 
lead to reconnended' program in^rovements. As evaluators become more 
es^erienced In assessing special features of community-based es^riential 
learning programs, as compared to school-based programs, lAey will also 
give attention to the diverse environments for learning that include the 
school, %iK>rk place, hmae and other community settings (such as 
participation in a Boy Scouts of America program) • Hhe environments 
become critical to stu^y in some cases because learning in one setting 
may reinforce or contradict learning in other settings. An exaa^le where 
confusion occurs is found when a young woman learns from her parents that 
her place as a future mother is in the home whil« at she is 

encouraged to explore nontraditional careers. 
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readftr mdftring how l» or sIm eonld |^i|sly con«ia*r *U of tlits« 
a^«cts of stiid«iit outoQM •valiMtion, %#t f«*l it is astf^ to radnd ' 
ftvalu«tO£« that, for worn projects, it is aqusUy iii«»ort4nt to ^sess 
progrsA affects on ti»^taff and oa participating comminity cesoorca 
persons. For exanplsr. it. has hean ft>ana, overV several years^ that • 

participating emoltyrg atria ma h^n^ri^u f-h^jy pftf^^i-pij^fl^lfpn in 

Siq^erience-Based Career BaJcation liot only a chance to help students bat, 
also, an ois^ortunity to infinDally screen potential wnployees, to become* 
better acquainted with the tanking of young people and c^in* prestige as 
a result of ea^laining their jbbs to interested young people (Oirens, 
Haenn vnA P^hrenb«Aer 1»75) . 

Bvaluation Unplewentation and Reporting , 

\ 

After engaging in the activities smested above as part of the 
evaluation* planning process it is necessary to toucih base with reality in 
recognising that the quickest way to an ineffective evaluation is to * 
attaint to evaluate every&ing. Criteria are li^portant for deteraining 
what variables to select for an intensive wtadsi, wliioh be «^lored in 
a more casual fashion and«^ich go b^ond the resources available. In 
selecting variables to be evaluatad, the evaluat«» needs to consider 
factors such ast 1) legislative or administratlva mandates related to 
the project, 2) the projeet*s objectives, 3) areas Ukely to affect 
program modifications, 4) prior fli^ifigs frcrnvralatid evaluations, 5) 
vaiiablw nUtwd to • tiwocatieal. •iplaiwtion of «lv a pragroi upclts. 
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v«£iafea«« and 7) variabifts r«lat»a to poli^ studies. 

^r^ons reviewing eveluations of h\%h a^ooX f^tUntM learning 
progrens h&Ve ofl»n conntented on the inadequate ceseprd^* designs 
enployed. Usually these criticisms reduce to a plea to en^lqy an , * 
. ♦^qgggjffl^P^J^ g^ggM^Jegign such as that involving pre and |>osfeteating^_ 
of an esip^iaental and contr<^ group, ^ "tthe pros and cons involving the 
appcopriateneas <tf such designs have been^^oroughly discussed elsewhere 
(Botouch pn, Cuba, 1S78)^ ihe point m wish to make here is that 
decisions regarding wtoprlate designs need to he considered in light of 
the develc^nental cycle of a project and^ the pui^ses for the ^ 
evaluation. In the case d momt new program, at least one year is 
needed to try out various delivery strategies, debug then and to 
estabj.ish a stable prograsi. Often a descriptive case stu^lr deaign is 
«ost ^^spropriate at this tine. A second prograsi ^se may be needed to 
test the abili^ of the prograai to achieve Its prosdsed outcasts,. In 
tiiis phase an experimental or guasi-ei^erimental design (Cai^ell and 
Stanly, 1913} may be psefia* Soae progras» progress to a third phase |a 
M<h the project's delivwry strategies have stabilised and the interest 
Shifts to considerations about expanding the prograai or disseminating tt 
to other districts. At this tiM people are usually Interested in the 
longer-terw impact of the prograsi on former students and in issues 
related to the traniporta|>ility of the |kn>jeot (i.e. program costs, staff 
training required and wsys of selling the concept to oUier educators and ' 

♦ 

41 



•the commil^). Them concerns sometiiMs lead to graduate foUow-i]^ 
studies and cost«-eff$ctivene8s studies. 

Programs based on ea^riential learning. In contrast to what James 
Coleman <1973) refers to as "knowledge assimilation" approaches, attempt 
to place young people in "^real world* environments In which they learn by 
directly engaging In activities and then reflecting on the meaning of 
their experiences. Su<A approaches are particularly appropriate for 
helping young people gain important conpetencles needed to be pOccessful 

would determine I 1) what learning took place, 2) how the learning 
occurred, 3) the different meanings that ^individuals attach to tl»ir , 
experiences and 4) the ability learners have gained to i^ly their new 
attitudes, knowledge and skills to new situations. Unfortunately, the 
authors have not discovered an evaluation report that addresses all four, 
areas. 

Since ei^rlentlal learning, as we have defined it, occurs in a real 
world environment rather than in the artificial environment often found 
In a classroom. It might be expected that naturalistic techniques would 
primarily be used to assess student performance, unfortunately, most of 
,the evaluation WB reviewed relied on paper and pencil instruments to 
assess learner outcomes, ^lied performance tests, designed to measure 
performance In a naturalistic settln^g, have not yet gained wide usage, 
ficanples where sudi acplled purformanoe tests would be useful are in 
coi^eting a job application, typing a business letter, demonstrating ^ 
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camnt life saving tr^atifients in it first aid class, participating in a 
jo;> Interviw simulation and in aiplying reading and nathenatics to 
• everyday lork problems. 



Examples of applied perfomance tests evolving from an experiential 
education program can be found in the Experience-Based Caret. Education 
n?rogram"(BBCE) developed through the ~N<^^ Regional Educational 

Laboratory. As a part of this EBCB program students master '•surviyiil 
skills** identified the local coiwaunity as essential for successful 
adult life in that community. At the EBCB demcxistration site in Tigard, 
Oregon* for example, the community representatives identified the 
following 13 competencies s transacting business on a credit basisi 
maintaining a dteckfing account in good caderi providing adequate 
insurance for self, family and possessions i filing state and federal 
income taxi budgeting time and money effectively, maintaining good 
pliysical health and making appropriate use of leisqre timei responding 
Wropr lately to fire,; police and physical health emergenciesi 
participating in the electoral processi understanding the basic structure 
and function .of local, state and federal governmenti eaeplalnlng one's 
legai rights and re^onsibilities} making appropriate use of pi^llc 
agenciesi making application for enplcyment and successfully holding a 
jobi and operating and maintaining an automobile. In most of these 
coaqpetencles, procedures have been Identified for relevant volunteers of 
the conmunlty to certify successful cospletlon of the conpeten^. Vot 

t 

exainple, a bank official certifies the cdiecklng account competency -i^hile 
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m intermix s«ryio« agent cftrtif 1m tht tax conpatancy aaploylag 

actual procedures and foriss used ty adults In the ooamunl^. 

In addition to applied performanoe measures, a nuiebee o£ otiier nontesting 
evaluation techniques have been used ty educator| interested in 
-evaluating student or program .outcomes of es^erlential learning 
programs. These techniques include analysis of anecdotal recor4st 
checklists* content analysis of documents, drawings, interviews, ^ 
journals, observations, peer nominations, photograpliy, ^slcal traces, 
que8ti<»uiaires, rating scales, role play, self inventories, simulations 
and tape and videotape recordings. . Although each of these techniques has 
its limitations, increased confidence in_ findings results when two or 
' more evaluation techniques used to measure the same dimension produce 
supporting evidence* 

V 

So far we have presented a frameworlc for conceptualising, designing and 
implementing evaluations of experiential learning programs. K final 
consideration we wish to discuss is that of reporting evaluation findings. 

Svaluation reports of expat lent ial learning programs have seldom 
discussed program findings in relation to the cdiaracteristics of the 
learners involved. Seldom is it shown whether program delivery 
strategies have a differential effect depending on the learner's sex, 
race, age, socioeconomic status or general adiievement level. Likewise 
it would be useful, i^en possible, to relate experiential learning 
characteristics to progra* delivery strategies and student outcomes. For 

4 I 
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«c«^«« posltlv* MinfdroftiMiit comnunlty rftaoiircft p«opl« eneountftred 
Ift career «3^oration« mt^ h% m k«y influence on mem youth in deeiding 
which particular caraar fiald to antajc, \ Sucdi finding* can help us 90 
bayond laaming that a studant has am a caraar choilca, to undarstanding 
predominant influences that. led t^^.^^i^^earning* 



Statistical reports seldoM communicata the flavor of what escperiential 
elements or delivery aysteiw %«ere particularly effective or ineffective 
in contributing to one or acre learner outcomes. Carefully planned and 
documanted student case stu^ procedures have ba^ developed 
ifFahrenbaGher, Owens and Haannr in process) have been develcsped that 
complanent statistical findings in conveying to staff, students, and 
community people the. individualized treatment that should be part of an 
experiential learning program. 
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III. POLICY SBCOMtDOMXIONS 
The follOiriii9 poliqy JC«cowi«na«tioiit have «Aer9«d in tlw pipcess of 

* » * 

as8«ssiii9 evaluations of «q>eriantlal l«arnlng |»roj«ct«* 

I. A re\rieif of tha lltaratui:* on es^rlential learning has 

uncovered a nurtb^c of theories such as tho«e itivolviag social 
learning, attribution, notivation, earaar daveloptent and 
parsonal/social devel<^]iient that could be used in better 
designing evaluations and interpreting experiential learning 

> 

program outcomes of education and training prograiw. Research 
should be supported that helps to identic ^ecif ic pos^lates 
from such theories that iiould help integrate and explain 
findings from experiential learning programs funded the 
^ .Department of Labor. 

2. ^theses of program evaluations of nork training program, 
career and vocational education programs and service learning 
^ programs have uncovered relatively few validated evaluation 
instrumants that can detect significant student gains on program 
outcomes. Research that mid indax and describe which 
instruments hav* been able to detect significant student growth 
on particular oul^comas tor various types of ypatii should be 
initlatad and disseminated to evaluators. Woe outcomes where no 
satisfactory instrumants axist, new instrument davelcpment 
efforts should be funded. . 
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Ihft disctissioii of «^roprl«t* •valuation dafigns to fit tha 
4aval<9«aataX cupela of adacatioa and wxk s^zograns indicatad 
that tha aaa.gsis ^muld natch tha dairalop«antal staga <rf tha 
program, if stabla progran dalivary stratagiaa ara ,to ba 
affectivaly assaasad, it baconaa assaiitial that tha Dapartmant 
of I«bor identic aona axaaplary progranB for a twor or thrae- 
yaar funding cycle. ' 

^Knowledga devalopmant activitias nead to ba carafully focused on 
a few quaationa for which data can feasibly ba obtained. ^ As 
evide^a aocunulatea on a particular issue, the focus of such 

I 

questions can be i^iftad to nair^ areas or issues of interest. 

Funding agencies which require program evaluation or knowledge 
develcq^ment activities should apaoify tha criteria for this 
activity prior to tha time of funding. These apaclfications 
should include such raqulremants for cross*^rograai studies as 
conpariaon groups, astiaatas of tha Mount and length of tasting 
tis» for youth, amount of staff time to be Involved and forms or 
instructions for provision of data* 

♦ 

Tha definition, of ai^ariantial laarning used in this study 
includad learning resulting from ui^lannad as wall as planned 
intarvantions* She Ufa aa^arianoa of many adults ^reveals that 
choica^ of Ufa styles and oaraars ara often influenced mwe hv 
unplanned critical ineidanta than fay a rational developmental 
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patt*rn« ife know litti«, hoMvef, about' irtiat conditions 
llk«]Y to cttsult in oritieal incidents for youi^ people and 

- hoir tbay learn to recognise th^ oc capitalise on sudi <^nce 

•» ^ _ J 

"events, ™l»e8^<*™in thl«nftM of eag^eriential learning coald be 

especially usefol to women and disadvantaged youth and suy 

Jprgduge greajgfir cl^aweg JotLiBipgoving.-JdieAr lives than i«ould 

continued research on traditional educational^ approaches. 

A number of elements of es^eriential learning are seen ty youth 
as iji^rtant for exceUent learning e3gc>erience8 in the 
CQwnunity. In-depth stu^ of some of thefe elements, such as 
de>^loping a sense of responsibility is essential in order to 
depict the constructt develop a t^^logy <^ levels of 
rei^Kxisibility, determine m^m of increasing this trait in young 
people and develop valid measures ?or assessing growth in this 
area* 

Studies of career decision making and attitudes toward work have 
generally centered in the s^moIs or in tha iiork place. Boir a 
young person receives su^orting or conflicting messages from 
different environments such as school,. :)ob site, home or other 
elements of the community is important to recognise if tie ' 
realise- that schools are not the sole location of learning, OOL 
funded research in tiiis direction may reveal i^ich environments 
are SKWt suited for proAieing which t^s of learning. 
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educAtioA and noric pco&raiui have indicated tha ioportance of 
, knowing di^erential pcogra^ oiitcones in ntUtion to 
eharacteristics of the particpatii!^ yoi^^ . nevertheless 

many evaluation reports still fail to differentiate program 
findings fcy the participants* sex, race, achievement level or 
socioeconomic level, !Ehe 'Department of i^bor guidelines for 
evaluation should request a breakout of evaluation report 
findings by relevant aiaracterlstics of the participating 
populati<Mi. 
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